A primary copy of data is generally a production copy or other “live” version of the data which is used by a software application and is generally in the native format of that application. Primary copy data may be maintained in a local memory or other high-speed storage device that allows for relatively fast data access if necessary. Such primary copy data is typically intended for short term retention (e.g., several hours or days) before some or all of the data is stored as one or more secondary copies, for example, to prevent loss of data in the event a problem occurred with the data stored in primary storage.
To protect primary copy data or for other purposes, such as regulatory compliance, secondary copies (alternatively referred to as “data protection copies”) can be made. Examples of secondary copies include a backup copy, a snapshot copy, a hierarchical storage management (“HSM”) copy, an archive copy, and other types of copies.
A backup copy is generally a point-in-time copy of the primary copy data stored in a backup format as opposed to in native application format. For example, a backup copy may be stored in a backup format that is optimized for compression and efficient long-term storage. Backup copies generally have relatively long retention periods and may be stored on media with slower retrieval times than other types of secondary copies and media. In some cases, backup copies may be stored at an offsite location.
After an initial, full backup of a data set is performed, periodic, intermittent, or continuous incremental backup operations may be subsequently performed on the data set. Each incremental backup operation copies only the primary copy data that has changed since the last full or incremental backup of the data set was performed. In this way, even if the entire set of primary copy data that is backed up is large, the amount of data that must be transferred during each incremental backup operation may be significantly smaller, since only the changed data needs to be transferred to secondary storage. Combined, one or more full backup and subsequent incremental copies may be utilized together to periodically or intermittently create a synthetic full backup copy. More details regarding synthetic storage operations are found in commonly-assigned U.S. patent application Ser. No. 12/510,059, entitled “Snapshot Storage and Management System with Indexing and User Interface,” filed Jul. 27, 2009, now U.S. Pat. No. 7,873,806, which is hereby incorporated herein in its entirety.
An archive copy is generally a copy of the primary copy data, but typically includes only a subset of the primary copy data that meets certain criteria and is usually stored in a format other than the native application format. For example, an archive copy might include only that data from the primary copy that is larger than a given size threshold or older than a given age threshold and that is stored in a backup format. Often, archive data is removed from the primary copy, and a stub is stored in the primary copy to indicate its new location. When a user requests access to the archive data that has been removed or migrated, systems use the stub to locate the data and often make recovery of the data appear transparent, even though the archive data may be stored at a location different from the remaining primary copy data.
Archive copies are typically created and tracked independently of other secondary copies, such as other backup copies. For example, to create a backup copy, the data storage system transfers a secondary copy of primary copy data to secondary storage and tracks the backup copy using a backup index separate from the archive index. To create an archive copy, a conventional data storage system transfers the primary copy data to be archived to secondary storage to create an archive copy, replaces the primary copy data with a stub, and tracks the archive copy using an archive index. Accordingly, the data storage system will transfer two separate times to secondary storage a primary copy data object that is both archived and backed-up.
Since each transfer consumes network and computing resources, the data storage system may not be able to devote such resources to other tasks. Moreover, the data storage system is required to devote resources to maintaining each separate index. In some cases, the archive index may be unaware of the other secondary copy and the other secondary index may be unaware of the archive copy, which may lead to further inefficiencies. Moreover, in some cases, in the event that an archive copy is moved or transferred (e.g., to another tier of secondary storage), the archive index may not be able to be updated to reflect the move or transfer. In such cases, the data storage system may be unable to use the stub to locate the archived data object.
Also, in conventional systems, archiving operations may require the transfer of large quantities of data during a single archive operation. For example, the retention criteria for an organization may specify that data objects more than two years old should be archived. On the first day of the organization's operation, it may be entirely unnecessary to archive any data, since the only data that exists at that point is newly created and thus ineligible for archiving. However, over the course of two years of operations, the organization may amass large quantities of data. Thus, when the first archive operation finally occurs, e.g., approximately two years into the operation of the organization, it may be necessary to transfer a large amount of the organization's data.
Additionally, backup, archive, and other secondary storage operations may unnecessarily preserve secondary copies of data created from primary data that has been deleted or is otherwise no longer being actively used as production data by a computing system, such as a workstation or server. Thus, secondary storage requirements may increasingly and unnecessarily bloat over time.
The need exists for systems and methods that overcome the above problems, as well as systems and methods that provide additional benefits. Overall, the examples herein of some prior or related systems and methods and their associated limitations are intended to be illustrative and not exclusive. Other limitations of existing or prior systems and methods will become apparent to those of skill in the art upon reading the following Detailed Description.